Skip to content

feat(skills): native Anthropic Skills upload + sync (closes #35)#36

Open
teslashibe wants to merge 5 commits into
mainfrom
feat/skills
Open

feat(skills): native Anthropic Skills upload + sync (closes #35)#36
teslashibe wants to merge 5 commits into
mainfrom
feat/skills

Conversation

@teslashibe
Copy link
Copy Markdown
Owner

Closes #35.

Wires the template into Anthropic's Beta Skills API so any fork can drop a SKILL.md folder (e.g. `~/.claude/skills/astrology-skill/`) and have it uploaded, persisted, and attached to per-user agents.

Summary

  • Backend (`internal/skills/`): one package, three files — `skills.go` (Service: List/UploadDir/UploadZip/SyncDirs/Delete + filesystem walker that preserves relative paths in the multipart upload), `zip.go` (single-skill zip extractor with path-traversal hardening), `handler.go` (Fiber CRUD).
  • Migration `00005_skills.sql` — `skills(id, anthropic_skill_id, name UNIQUE, description, source, version, timestamps)`. UNIQUE(name) makes re-syncs idempotent (UPSERT updates the existing row's Anthropic ID + version).
  • Provisioner: both `cmd/provision` (bootstrap, optional `SKILLS_AGENT_IDS` env) and `internal/agent/provision.go` (per-user) attach skill IDs. The agent toolset (bash + code-execution) is added only when at least one skill is configured — keeps the default permission surface unchanged for forks not using skills.
  • CLI `cmd/skills-sync`: walks `SKILLS_SOURCES` (comma-separated dirs), uploads every immediate child containing `SKILL.md`. Folders without `SKILL.md` are skipped with a warning, not aborted.
  • Mobile: Settings → Skills card → Skills screen with upload (zip via DocumentPicker), delete (with confirm), and sync. Empty state CTAs upload.
  • Docs: `docs/SKILLS.md` covers SKILL.md format, folder layout, sandbox constraints (no network → bundle wheels), and the REST surface.

Constraints worth re-flagging in review

  • Anthropic's API sandbox has no network and no `pip install`. Skills with native deps must bundle wheels (the astrology skill already does).
  • Per-agent 20-skill cap — `AnthropicIDs` selects the oldest 20 by `created_at`. Documented in `docs/SKILLS.md`.
  • Skills are workspace-wide (shared across everyone using the same `ANTHROPIC_API_KEY`); no per-team isolation in this PR.
  • Per request: no tests in this PR. Listed under "Out of scope" in feat(skills): native Anthropic Skills upload + sync #35; will land in follow-up.

Test plan

  • Backend builds (`go build ./...`).
  • Mobile typechecks (`tsc --noEmit`).
  • `make db-reset && make up` — confirm migration runs.
  • Set `SKILLS_SOURCES=~/.claude/skills`, run `make skills-sync`, confirm `astrology-skill` shows up at `GET /api/skills/`.
  • In the mobile UI, open Settings → Skills, confirm the list, hit delete, confirm row is gone from both Anthropic and the DB.
  • Start a new chat, ask the agent to "use the astrology skill to draft a chart for someone born today" — verify Claude reads SKILL.md via bash.

Made with Cursor

teslashibe and others added 2 commits May 9, 2026 10:52
closes #35 (backend slice)

- internal/skills: Service wraps Anthropic Beta.Skills (List, Upload,
  Delete) and persists skill_id/name/description/version in Postgres.
  UploadDir parses SKILL.md frontmatter, walks the folder, preserves
  relative paths via a namedReader so the multipart upload matches
  what the API expects.
- internal/skills/zip.go: rejects path-traversal entries and zips
  without exactly one top-level folder + SKILL.md.
- cmd/skills-sync: CLI driven by SKILLS_SOURCES that bulk-uploads
  every SKILL.md folder it finds; idempotent via UNIQUE(name) + UPSERT.
- internal/agent/provision.go: SkillIDsFn opt; per-user agents now
  attach the agent toolset (bash/code-execution) alongside their MCP
  toolset only when at least one skill is configured.
- cmd/provision/main.go: optional SKILLS_AGENT_IDS env to attach
  skills at bootstrap.
- 00005_skills.sql migration with UNIQUE(name) + index on anthropic_skill_id.
- /api/skills CRUD wired in main.go; Makefile target skills-sync;
  SKILLS_SOURCES + SKILLS_AGENT_IDS in .env.example.

Co-authored-by: Cursor <cursoragent@cursor.com>
closes #35 (mobile slice)

- mobile/services/skills.ts: typed client for /api/skills CRUD; uses
  expo/fetch for the multipart upload so we can build FormData.
- mobile/app/(app)/skills.tsx: list + upload (zip via DocumentPicker)
  + delete (with confirm) + sync button. Empty state CTAs upload.
- settings.tsx: link to Skills card.
- _layout.tsx: register hidden 'skills' route so deep links work.
- docs/SKILLS.md: SKILL.md format, folder layout, sandbox constraints
  (no network, bundle wheels), make skills-sync workflow, REST surface.
- mobile/package.json: expo-document-picker dependency.

Co-authored-by: Cursor <cursoragent@cursor.com>
@teslashibe
Copy link
Copy Markdown
Owner Author

Forensic Audit — Skills MVP vs #35

Per .cursor/rules/issue-audit-user-stories.mdc. Done after the implementation, before review.

1. Findings (severity-ordered, evidence first)

HIGH

H1. Re-uploading a changed skill orphans the old Anthropic skill.
backend/internal/skills/skills.go:149 calls Beta.Skills.New(...) every time UploadDir runs and then UPSERTs by name. When a skill with the same name already exists in our DB, the row's anthropic_skill_id is rotated to point at the new skill — but the old skill is never deleted from Anthropic. Over time, every re-sync orphans one skill per change. The SDK exposes Beta.Skills.Versions.New(skillID, ...) for the correct flow (push a new version onto the existing skill).
Fix: on UPSERT collision, fetch the existing anthropic_skill_id first; if present, call Versions.New instead of Skills.New. Fall back to Skills.New only when no prior row exists.

H2. Newly uploaded skills don't reach already-provisioned per-user agents.
internal/agent/provision.go:EnsureForUser caches the per-user anthropic_agent_id in users and createAgent (which reads SkillIDsFn) only runs on the FIRST call. After the first chat, uploading a skill via the UI has zero effect for that user — the cached agent has the old skill list forever. The SDK exposes Beta.Agents.Update(agentID, ...) with a Skills field (betaagent.go:1977 "Skills. Full replacement.") that solves this.
Fix: when Service.UploadDir/Delete succeed, broadcast a "skills changed" signal that triggers Agents.Update for every cached user. Cheaper alternative: add a skills_revision column to users, compare on each session create, and re-create-or-update if stale.

H3. Upload endpoint inherits Fiber's 4 MiB default body limit.
cmd/server/main.go:78 constructs Fiber without setting BodyLimit. Skills with bundled wheels (the astrology skill ships pyswisseph wheels — ~5 MiB) will 413 before even reaching the handler. AC2 ("uploads via API") will fail on the canonical example skill.
Fix: raise BodyLimit to ~64 MiB, or set BodyLimit per-route via Fiber's per-route config on /api/skills.

MEDIUM

M1. AnthropicIDs silently truncates at 20.
skills.go:84 selects LIMIT 20 to honour Anthropic's per-agent skill cap, but never logs when it truncates. A workspace with 25 skills will quietly attach the oldest 20 to every new agent and the operator gets no signal. Documented in docs/SKILLS.md but not surfaced at runtime.
Fix: when len(rows) == 20 and a SELECT count(*) returns more, log a warning with the truncated names.

M2. Brittle 404 detection on Anthropic delete.
skills.go:117 does strings.Contains(err.Error(), "404") to decide whether to swallow the Anthropic error and clean the local row. The SDK error type carries a structured status code (*anthropic.Error.StatusCode); string-matching the message is fragile against locale/i18n or SDK upgrades.
Fix: type-assert to *anthropic.Error and check StatusCode == 404.

M3. make skills-sync env loading.
Makefile:skills-sync does source backend/.env 2>/dev/null || true — but bash will still fail on lines with empty values like RESEND_API_KEY= if the user's env file has any quoting quirks. Reliable in zsh, brittle elsewhere.
Fix: rely on godotenv.Load() inside cmd/skills-sync/main.go (already present at L21) and drop the shell-level source from the Makefile.

LOW

L1. Sync handler logs nothing.
handler.go:94 passes a no-op log function, so server-side bulk syncs from the API leave no trail. The CLI logs to stdout. Inconsistent observability.

L2. Upload handler reads entire file into memory.
handler.go:53 does io.ReadAll(f) then passes the byte slice to UploadZip. For a 50 MiB skill bundle that's a transient ~150 MiB allocation (file → bytes → zip reader). archive/zip accepts an io.ReaderAt so we could write to a temp file and stream.
Trade-off: minor; "lean" beats "optimal" in MVP.

L3. Custom YAML frontmatter parser is reinventing a wheel.
skills.go:222 readSkillMD walks the --- delimiters by hand. gopkg.in/yaml.v3 was added to go.mod for the actual parse — goldmark-meta or even the yaml.v3 decoder against a 2-document stream would be standard.
Trade-off: acceptable; the hand-roll is 20 lines and works.

L4. expo-document-picker lacks a peer-version constraint.
mobile/package.json got expo-document-picker from npm install --silent which writes the latest. Expo SDK 55 wants a specific version — npx expo install would have respected that. Dev experience hazard, not a runtime bug.

2. Gaps vs issue intent

Issue intent Status Notes
make skills-sync from local paths ✅ done cmd/skills-sync + Makefile target.
Re-sync is idempotent ⚠️ partial UPSERT on name ✓; but H1 orphans the old Anthropic skill so it's idempotent in our DB only.
Folder w/o SKILL.md is skipped with warning ✅ done Service.SyncDirs + os.Stat check.
Upload via multipart .zip ✅ done handler.upload + unzipSingleSkill.
Missing SKILL.md → 400, no Anthropic call ✅ done unzipSingleSkill errors before UploadDir runs.
Provisioner attaches code-execution + skill IDs ✅ done — for NEW agents H2: existing per-user agents never get updated.
Settings → Skills lists name/description/source/delete ✅ done mobile/app/(app)/skills.tsx.
Delete removes from Anthropic AND DB ✅ done Best-effort, swallows 404 (see M2).
ANTHROPIC_API_KEY missing → CLI exits early ✅ done cmd/skills-sync/main.go:23 log.Fatal.
Anthropic 4xx → no DB row + error surfaced ✅ done UploadDir errors before INSERT.

The two real gaps are H1 and H2. Everything else either passed or has a low-severity caveat documented above.

3. User stories (audited)

  • As a template forker, I want to point SKILLS_SOURCES at a directory of skill folders and run make skills-sync, so that all my custom skills are uploaded to Anthropic. — MET (with H1 caveat for re-runs).
  • As an end-user, I want to see which Skills my agent has installed and remove ones I don't need. — MET.
  • As an end-user, I want to upload a new Skill folder/zip from the UI, so that I can add capabilities without redeploying. — PARTIALLY MET; H3 will reject any skill with bundled wheels until BodyLimit is raised, and H2 means even successful uploads don't reach my existing agent until I sign out and back in.
  • As a backend operator, I want skill IDs persisted in our DB so the UI doesn't round-trip Anthropic on every page load. — MET.

4. Acceptance criteria (re-stated, testable)

Sync from filesystem

  • AC-1.1 Given SKILLS_SOURCES=<dir>; when make skills-sync runs; then every <dir>/*/SKILL.md is uploaded and GET /api/skills/ returns those rows.
  • [⚠️] AC-1.2 Given a skill with the same name already exists; when re-running; then the local row UPDATES in place. Pass for our DB; FAIL on Anthropic side per H1 — old skill not cleaned up.
  • AC-1.3 Given a folder lacks SKILL.md; when sync walks it; then it's skipped with a logged warning and the run continues.

Upload via API

  • [⚠️] AC-2.1 Given an authenticated user; when they POST a multipart <5 MiB .zip; then the response is 201 with the new skill JSON. For >4 MiB zips → fails per H3.
  • AC-2.2 Given the zip lacks SKILL.md at root; then the server returns 400 and does NOT call Anthropic.
  • AC-2.3 Given the zip has more than one top-level folder; then the server returns 400 with "exactly one top-level folder".
  • AC-2.4 Given a path-traversal entry (e.g. ../etc/passwd); then the server returns 400 with "escapes root".

Provisioner attaches skills

  • AC-3.1 Given ≥1 skill in DB; when a NEW user creates their first session; then the Anthropic Agent has the agent toolset (bash + code-exec) AND those skill IDs.
  • [❌] AC-3.2 Given a user has already chatted (cached agent_id); when a skill is uploaded; then the user's NEXT session uses an agent that knows about the new skill. FAIL per H2 — cached agent is never refreshed.

Mobile UI

  • AC-4.1 Given I open Settings → Skills; then I see name, description, source, and a delete affordance per row.
  • AC-4.2 Given I tap delete and confirm; then the row is removed from Anthropic + DB and disappears from the list.

Negative paths

  • AC-5.1 Given ANTHROPIC_API_KEY=""; when make skills-sync runs; then exit code is non-zero with a clear log line and no Anthropic calls were made.
  • AC-5.2 Given Anthropic returns 4xx on upload; then no DB row is written and the caller sees the error.

5. Risks and follow-up actions

Risk Likelihood Mitigation
Anthropic Skills API beta header changes Med SDK upgrade absorbs it; pinned at v1.37.0 in go.mod.
Skill bundle exceeds Fiber body limit (H3) High for canonical astrology skill Raise BodyLimit; ship as part of this PR or a follow-up before users hit it.
Orphaned Anthropic skills on re-sync (H1) High after first re-sync Implement Versions.New flow before announcing the feature externally.
Stale per-user agents miss new skills (H2) High after first upload Add Agents.Update call from Service.UploadDir.
Path-traversal regression Low (covered) unzipSingleSkill guards; consider a regression test in the follow-up tests PR.

Recommended follow-up tickets

  1. fix(skills): use Versions.New on re-upload — H1.
  2. fix(skills): refresh cached per-user agents on skill change — H2.
  3. fix(skills): raise Fiber BodyLimit for /api/skills uploads — H3.
  4. chore(skills): structured 404 detection on Anthropic delete — M2.
  5. chore(skills): log truncation when AnthropicIDs hits the 20-cap — M1.
  6. test(skills): table-driven coverage — explicitly deferred per spec; opens after the H-level fixes land.

Bottom line: the MVP meets 8/10 ACs as written. The two failing ACs (1.2 partial, 2.1 and 3.2) are real and high-severity but each is a targeted ~20-line fix using SDK methods we already pull in. None of them require redesign — they're bugs introduced by writing the simplest version first.

teslashibe and others added 3 commits May 10, 2026 08:14
…ome pattern)

Demonstrates how to build a skill whose compute can't fit in Anthropic's
sandbox (native deps, data files, side effects). Splits the skill in two:

- backend/internal/astrology + internal/mcp/platforms/astrology.go:
  pure-Go sun-sign computation registered as the astrology_birth_summary
  MCP tool. Stub for moon/ascendant/houses with comment pointing at
  mshafiee/swephgo for full Swiss Ephemeris accuracy.
- skills/astrology/{SKILL.md, reference.md}: tiny skill that tells
  Claude to gather birth data, call the MCP tool, read reference.md
  for interpretation tables, and compose a tight reading.

docs/SKILLS.md gains an 'Authoring skills for this template' section
that documents the pattern, with the astrology skill as the worked
example. Also calls out the constraints surfaced during live testing:
no nested archives (.whl/.zip/.tar/.tgz/.gz are silently skipped) and
display_title uniqueness on re-upload (issue #35 follow-up to fix
via Versions.New).

Co-authored-by: Cursor <cursoragent@cursor.com>
Surfaced during live testing of the astrology skill upload:

- ~/.claude/skills/foo is canonically a symlink at the real repo. The
  sync's e.IsDir() check returned false on symlink entries; switch to
  os.Stat (follows symlinks) and resolve via filepath.EvalSymlinks
  before walking, since filepath.Walk does not follow symlinks itself.
  Preserve the visible folder name in the uploaded paths so the skill
  still appears as 'foo' to Anthropic even when the symlink target is
  named differently.
- Anthropic's API rejects 'Skill cannot contain nested zip files'.
  Skip .whl, .zip, .tar, .tgz, .gz extensions in openSkillFiles so
  the rest of the skill still uploads. The astrology skill ships
  pyswisseph wheels under scripts/wheels/ that triggered this.

Co-authored-by: Cursor <cursoragent@cursor.com>
closes the H1/H2/H3 + M1/M2/M3 findings from the forensic audit on PR #36
and three more constraints surfaced during live testing.

H1 — display_title uniqueness on re-upload (audit + live)
  Anthropic rejects Skills.New when a skill with that display_title
  already exists. Service.UploadDir now looks up the prior anthropic
  skill ID for that name and uses Beta.Skills.Versions.New on
  collision; only first uploads call Beta.Skills.New.

H2 — newly-uploaded skills don't reach cached per-user agents
  refreshAgentsAsync fires after every successful upload/delete: it
  enumerates every users.anthropic_agent_id and pushes the current
  skill list via Beta.Agents.Update. Fired-and-forgotten so upload
  latency stays low; failures are logged.

H3 — Fiber's 4 MiB default body limit rejects wheel-bundled skills
  Bumped fiber.Config.BodyLimit to 64 MiB (matches Anthropic's
  per-skill cap). Verified live with a 6 MiB upload that previously
  413'd.

NEW: Cannot delete skill while versions exist
  deleteAllVersions enumerates every version via the Versions service
  and deletes each before calling Skills.Delete. 404s on individual
  versions are swallowed.

NEW: SDK helper BetaManagedAgentsSkillParamsOfCustom omits required Type
  Returns 400 'skills[0].type: Field required'. Wrapped in skills.SkillParams
  which sets Type to BetaManagedAgentsCustomSkillParamsTypeCustom.
  Provisioner refactored to use it.

NEW: Anthropic's cloud cannot reach loopback MCP URLs
  createAgent now fails fast with an actionable hint pointing at
  ngrok/cloudflared instead of waiting for Anthropic to return a
  generic 400.

M1 — silent truncation at 20-skill cap
  AnthropicIDs now logs the names of skills it dropped.

M2 — brittle 404 detection
  Added is404 helper using errors.As(*anthropic.Error{}); replaces the
  strings.Contains heuristic in Delete + deleteAllVersions.

M3 — bash-source brittleness in Makefile
  Dropped the 'source backend/.env' incantation; the CLI uses
  godotenv.Load() so the Makefile target is one line now.

Bonus: Astrology binding sets NoCredentials: true so the demonstration
tool doesn't trigger the credentials guard for credentialless plugins.
skills-sync waits 2s before exiting so the async refresh goroutine can
drain (was logging 'closed pool' in CLI output).

Co-authored-by: Cursor <cursoragent@cursor.com>
@teslashibe
Copy link
Copy Markdown
Owner Author

Live Test Results — All Audit Findings Resolved

Tested every API + Web route on the live stack. All audit findings (H1/H2/H3 + M1/M2/M3) plus three additional Anthropic API constraints surfaced during testing are now fixed and verified.

Audit fixes

Finding Before After
H1 display_title uniqueness on re-upload Re-running make skills-sync returned 400 `Skill cannot reuse an existing display_title` UploadDir looks up the prior anthropic_skill_id and uses Beta.Skills.Versions.New; first uploads still use Beta.Skills.New. Verified: same skill ID, new version on each re-sync.
H2 cached per-user agents missed new skills Uploading a skill only reached new users; existing users had a stale cached anthropic_agent_id refreshAgentsAsync fires after every upload/delete, enumerates users.anthropic_agent_id, pushes the new list via Beta.Agents.Update. Fire-and-forget; failures logged.
H3 Fiber 4 MiB body limit Skill bundles >4 MiB returned 413 before reaching the handler BodyLimit: 64 * 1024 * 1024. Verified with a 6 MiB upload — HTTP 201.
M1 silent 20-skill truncation No log when AnthropicIDs capped at 20 Logs the names of dropped skills with the cap value.
M2 brittle strings.Contains(err.Error(), "404") Fragile across SDK upgrades / locales New is404 helper uses errors.As(*anthropic.Error{}) then checks StatusCode == 404. Used in both Delete and deleteAllVersions.
M3 brittle source backend/.env in Makefile Failed on env files with quoting quirks Dropped — CLI uses godotenv.Load() already; Makefile target is one line now.

Additional Anthropic API constraints surfaced (live testing)

Constraint Symptom Fix
Cannot delete skill with existing versions DELETE on any re-uploaded skill returned 500 deleteAllVersions enumerates via Versions.ListAutoPaging and deletes each before calling Skills.Delete.
SDK helper BetaManagedAgentsSkillParamsOfCustom omits the required Type field Beta.Agents.New returned 400 `skills[0].type: Field required` Wrapped in skills.SkillParams that sets Type: BetaManagedAgentsCustomSkillParamsTypeCustom. Provisioner uses it.
Anthropic cloud rejects loopback MCP URLs (localhost/127.0.0.1/::1) Local-dev session creation hung 750ms then 500'd with a confusing 400 from Anthropic createAgent now fails fast with: MCP_PUBLIC_URL/APP_URL resolves to loopback (localhost); for local dev, expose via ngrok or cloudflared and set MCP_PUBLIC_URL to the public URL
Anthropic API sandbox rejects bundled .whl/.zip/.tar/.tgz/.gz "Skill cannot contain nested zip files" openSkillFiles skips these extensions silently. (Already in earlier commit.)
Symlinked source dirs (~/.claude/skills/foo) silently skipped e.IsDir() returns false for symlinks Switched to os.Stat + filepath.EvalSymlinks before walking. (Already in earlier commit.)

Live test results

=== AUTH ===
  POST /auth/login → JWT (201 chars)

=== API READ ROUTES (auth) ===
  200  /api/me                         (195 bytes)
  200  /api/teams/                     (303 bytes)
  200  /api/skills/                    (633 bytes)
  200  /api/agent/sessions             (346 bytes)
  200  /api/platforms                  (634 bytes)

=== UNAUTH GUARDS (expect 401) ===
  401  /api/me
  401  /api/skills/
  401  /api/agent/sessions
  401  /api/teams/

=== SKILL UPLOAD VALIDATION (expect 400) ===
  400  empty POST          → "missing 'file' upload (zip of the skill folder)"
  400  notazip.txt         → "upload must be a .zip"
  400  empty-skill.zip     → "zip root empty-skill does not contain SKILL.md"

=== EXPO WEB ROUTES ===
  200  /  /chat  /settings  /skills  /platforms  /teams  /capture

Skills-specific E2E

  • Sync ~/.claude/skills/astrology-skill/ (symlinked to ~/teslashibe/astrology-skill) → uploaded on first run, Versions.New on re-run (same skill_01... ID, new version timestamp). ✓
  • Multipart upload of a 3.5 KiB skill zip → HTTP 201, UPSERT updates existing row. ✓
  • 6 MiB bodyHTTP 201 (would have been 413 before H3 fix). ✓
  • Delete a versioned skill → HTTP 204 (was 500 with "Cannot delete skill with existing versions"). ✓
  • Direct MCP tool call astrology_birth_summary({date: 1990-08-15, time: 14:30, timezone: America/New_York, place: NYC})Sun in Leo · Fire · Fixed · Sun-ruled. ✓ Engine works end-to-end.

What was NOT tested live (with reason)

  • Agent → MCP tool round-trip from a real chat: Anthropic's cloud cannot reach localhost. To test this end-to-end locally, run ngrok http 8080 and set MCP_PUBLIC_URL=https://<id>.ngrok.app before bringing up the stack. The fail-fast hint added in this commit makes the constraint visible immediately instead of mid-session. In production this is a non-issue.

Files changed in this push

backend/cmd/server/main.go                   |   8 +- (BodyLimit 64MiB)
backend/cmd/skills-sync/main.go              |   8 ++ (drain refresh goroutine)
backend/internal/agent/provision.go          |  20 ++ (loopback guard, SkillParams)
backend/internal/mcp/platforms/astrology.go  |   7 +  (NoCredentials: true)
backend/internal/skills/skills.go            | 196 +++ (Versions.New + delete cascade + refreshAgentsAsync + is404 + SkillParams + truncation log)
Makefile                                     |   3 +- (drop bash source)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): native Anthropic Skills upload + sync

1 participant